WebdriverIO extensive reporting and more with test management

Test automation is essential for testers to keep up with the increasing demands of faster delivery within shorter timelines and optimal software quality. For this purpose, the QA community implements many progressive testing tools. And their number is growing. Let’s see below, we will walk you through a test project with one of them 🤖.

What is WebDriverIO?

WebDriverIO is the leading test automation framework for both web and mobile applications. It is written in JavaScript and runs on NodeJS.

WebDriverIO – is an independent framework, which means any corporate interests don’t influence which way it should be developed and where it is supposed to go. In fact, it is a unique tool in comparison to many automation tools. WebDriverIO is a definitely open-source project that evolves within open governance and is owned by a non-profit OpenJS Foundation.

A vast NodeJS community around the world helps the framework grow with new capabilities and cares about its stability. Followers communicate within Gitter Also you can ask your ques within Stack overflow. There are more than 500+ discussions there. You can see many useful links on the official Resource page in one place.

Automation testing with WebdriverIO

WebDriverIO is a flexible and powerful automation framework with good documentation, functionalities a variety of plugins and good support as we just mentioned before.

📄 Extract from official Docs, it emphasises wdio is designed to be:

• Extendable – adding helper functions, or more complicated sets and combinations of existing commands is simple and really useful
• Compatible – testing framework coded with wdio possible runs on the WebDriver Protocol for true cross-browser testing as well as Chrome DevTools Protocol for Chromium based automation using Puppeteer.
• Feature Rich – the variety of built-in and community plugins allows QA teams to easily integrate and extend your setup to meet the requirements of Agile SDLC.

Thus, WebDriverIO is perfect choice to automate E2E tests for modern JavaScript web applications. As it is built in NodeJS, testers and developers can automate their test scripts using JavaScript, as well as with TypeScript (it is up to you). And at the same time, they can do it in a fast and reliable way. The tests performed on WebDriverIO are simple and concise.

💡Especially this applies to web applications designed:

• Modern frontend frameworks – such as React, Vue, Angular, Svelte etc.
• Hybrid mobile applications – empowered by iOS and Android applications using Appium, for instance, the Ionic JS framework or native mobile applications running in an emulator/simulator or on a real device.
• Native desktop applications – designed with Electron.js for Windows and macOS.
• Unit or component testing of web components in the browser.

WebDriverIO is appropriate for both Behavior Driven Development (BDD) and Test Driven Development (TDD) which one more time makes opt for it for automation testers.

To start with WebDriverIO, you need just 2 CMD commands

👀 Look at the code extract and check out by yourself how you can initialize WDIO testing project and run your first wdio auto test really per a few clicks:

npm init wdio . //initialize WebDriverIO 
npm init wdio ./path/to/new/project /* create a project with the basic structure
in a designated folder */
yarn create wdio ./path/to/new/project //create a project with yarn packages
npx wdio run ./wdio.conf.js // run tests

🔥 Amazing! Great implementation of the idea by wdio development team, isn’t it?

The first command downloads the WebdriverIO CLI tool that wizard you to configure your test suite by prompting a set of questions that guides you through the setup. You can pass a –yes parameter to pick a default set up which will use Mocha with Chrome using the Page Object pattern. CLI also provides information on all available third-party framework adaptations, reporters, diverse services, etc.

And the second command runs your tests, by default it is the first login test. More detail with adding parameters running test scripts in Docs

Why do many QA teams choose WebDriverIO?

Eventually, in one voice, QA engineers note advantages, and we made an attemptе to categorize them. Keep these in mind when looking for a solution for your testing framework.

Benefits of test automation with WebdriverIO:

• Parallel test execution and cross-browser test automation. WebdriverIO is based on Web Standards, is supported by all browser vendors and guarantees a true cross-browser testing experience with all major browsers Chrome, Firefox, Edge, Internet Explorer, and Safari.
• Supports integrations to Chrome DevTools and Google Lighthouse, which are widely used developer tools.
• Diverse selectors and auto-waiting mechanism are able to cover common user interactions.
• Convenient test runner comes with a CLI play great utility tandem thanks to their configuration potency. This contributes to extendability due to the possibility of adding helper functions, building more complex combinations of available commands and even the ability to create your custom commands.
• Integration with commonly used Cloud and CI/CD tools. Can be used locally, as well as in the cloud, through services like Sauce Labs, Lambdatest or BrowserStack.
• Carries multiple reporters like Junit, Allure, Spec, HTML, Video, JSON and custom reporting to your convenience.

Sync WebdriverIO tests with manual testing

Since WebdriverIO is a loved tool by many test engineers, the testomat.io team didn’t want to be left out and implemented integration with it as a part of the Test automation Integrations group.

Seamless WebdriverIO integration helps many Agile teams bring their automation test management to the next level. Makes testing and debugging of the testing framework scripted with WebdriverIO a lot more efficient for our customers. Automation QA engineers are able to easily import their tests into the test management system and make them visible to the whole team.

Finally, they can collaborate with non-tech teammates in one space before they difficulty achieved a common understanding.

Let us show you how everything works 👀

Getting started WebdriverIO from example

Our team prepared a bunch of Demo projects for success starting with the testomat.io TCMS. Besides WebdriverIO following through the link, you can find Cypress, Playwright, Cucumber, Jest, CodeceptJS and other examples of testing frameworks. Download the needed project there and try how it works by yourself.

These are the steps you need to follow to run the example in testomat.io test management:

Before exploring how to run WebdriverIO tests with test management, first of all, the prerequisites mentioned below have to be met.

Prerequisites for WebdriverIO testing framework

Check the NodeJS version, enter the following commands:

$ node -v

Install NodeJS or update the NodeJS, by entering the following command in project directory:

npm install --save-dev

If everything goes without errors, let’s head Step by Step!

#1 Create a new test project

Start by Sign Up or Sign In to the TMS, and then create your test project by honouring the system tips. We begin by writing the project name and choosing a classical type of project, as in the picture below. You may configure your WebDriverIO test project using different settings according to your testing frameworks, such as Chai or Mocha, JavaScript language or TypeScript, classical project or BDD. You will bump all these options as we go along.

After you create your project, get into it. Select Import Automated tests on the dropdown menu in the top right corner. Thanks to the Advanced Importer you can import your automated tests to the test management system with a few clicks and make them visible there for the whole team.

Configure import according to your testing project. For instance, we chose a project using WebDriverIO with Mocha and JavaScript option.

#2 Generate API key and import autotests

Our TCMS is a wise tool. TCMS recognizes your laptop OS and generates a command with API key according to it. You should copy this command and execute it on CMD. This one uploads your WebdriverIO tests from source code to test management.

Look at the console, how the importer analyzes code accurately and displays the number of founded tests in the test framework. If you enter your project, you will see these tests. You can view their structure by expanding the tree. You may enter your suites and check test cases inside.

Now you can synchronize your manual tests alongside automated tests scripted with WebdriverIO in one place and get aggregated test result reports and analytics. Even the number of tests increases significantly – custom labels and tags help organize them conveniently. Sort, Change order, Filter and Search test cases quickly in scale.

Please note: with each new import, the intelligent importer will analyze test cases present in it or not. If there are no new test cases in the test management system yet, you will add new ones. These tests will be marked as out of sync by TCMS. Vice versa, if they are not in code, they will be marked as detached test cases. You might check them and delete unnecessary tests.

#3 How do I create a report in Wdio?

Track test results to test management with our advanced reporter. Test management testomat.io reporter connects with WebdriverIO an open-source test automation tool. Copy and paste this command into CMD to install reporter:

npm i --save-dev @testomatio/reporter

#4 Configure .conf file:

We suggest adding our reporter (test management plugin) along with existing reporters in wdio.conf.js. For this purpose copy configuration suggested by TMS to the project which can find out in the DEMO project as well.

const testomatio = require('@testomatio/reporter/lib/adapter/webdriver');

reporters: [
[testomatio, {
apiKey: process.env.TESTOMATIO
}]
],

#5 Execute WebdriverIO scripts

You are ready now to run your WebdriverIO tests with test management. Execute WebdriverIO scripts and receive results synced in test management.

Run the following command from your project folder:

TESTOMATIO=<API key> npx start-test-run -c 'npx wdio'

Once the tests have finished running, the CLI will generate a report link. Test results will appear through the console on the test management dashboard. Share this test result link with stakeholders by email or messengers. Choose whom available detailed report. The type of public report does not contain security-sensitive info.

We have performed tests oft, and each of these runs saves some historical data.

We can analyze the results of running tests using advanced analytics. The following testing metrics are available on the Project Analytics dashboard and Global Analytics dashboard. Our team are currently working on expanding with new widgets.

A key characteristic of the CI/CD pipeline is the use of automation to ensure code quality. Rich reporting and real-time monitoring are essential to understand and quickly address any problems, especially matters to CI/CD pipeline. Collection and gathered statistics and metrics are necessary. Pipeline successes and failures, as well as how long each test ran can be measured by key indicators and stored in Runs history over time.

Analytics shows the history of your test runs and failures, flaky tests, slowest tests, never run tests as well as test automation coverage of your test scope.

Statistics are gathered by collecting all available test run results for the project and tags which are marked test cases. Analytics data is available on a paid subscription. You are able to filter and sort your test cases from a low to high rate or by tags.

In detail, we describe each step precisely and in order in official Documentation as well. Check it for more options.

Running WebdriverIO test on CI\CD

QAs can run their tests in the cloud and on CI\CD besides on a local machine. CI\CD test execution carries out testing delivery much faster.

Moreover, if you set up a CI\CD integration connection, you can run your tests immediately from the test management system, and non-technical specialists directly from Jira.

So, our test management solution support running wdio tests on CircleCI, AzureDevOps, Jenkins, GitHub, Gitlab, Bamboo etc.

• Running WebdriverIO test on GitHub Actions CI
• End-to-end testing with GitLab CI\CD and WebdriverIO
• WebDriverIO Integration With Cucumber
• Test Automation with CircleCI & Reporting
• Automating Tests in Azure Pipelines

Comparison table: WebdriverIO VS Cypress VS Selenium

Most of today’s software development job openings request JavaScript knowledge to be used for test automation, given the many advantages of JavaScript automation tools like WebdriverIO, Cypress, TestCafe, Nightmare, CodeceptJS, Playwright and BDD Cucumber.

As you can see by yourself, our testing tool offers a great alternative to test reporting for all these test frameworks, and much more it helps you keep your tests in a clear organizational structure.

The test case management tool testomat.io has a free subscription forever, so you should not worry you lost your tests. They will be stored securely.

Let me know what you think about this setup, and/or if you have any questions, please do not hesitate to post them below. Thank you for taking the time to read my article.

The content is available also in video format if you convenient reproduce the steps to visualize your tests and get an informative report by watching.